-
Notifications
You must be signed in to change notification settings - Fork 6.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ZVM : New-Generation Type 1.5 RTOS Virtualization Solution (Hypervisor) #84123
base: main
Are you sure you want to change the base?
Conversation
94d9fc3
to
3ada286
Compare
arch/arm64/core/zvm/timer.c
Outdated
* This needs to be done when the vcpu is created. The step is below: | ||
* 1. Init vtimer and ptimer register. | ||
* 2. Add a timer expiry function for vcpu. | ||
* 3. Add a callbak function. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
callbak -> callback
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, it has been corrected.
What is changed? - Added a new qemu_max board for QEMU emulation. This includes the addition of board-specific files such as Kconfig, defconfig, board.yaml and DTS files under `boards/qemu/max`. - Added support for multi-core SMP with QEMU through the `qemu_max_qemu_max_smp` configuration and associated DTS and YAML files. - Updated SoC-specific configurations under `soc/arm/qemu_max`, including Kconfig, SoC-specific settings, and memory regions in `mmu_regions.c`. - Added device tree include file `qemu-virt-max.dtsi` for enhanced compatibility with the qemu_max board. Why do we need this change? - The qemu_max board supports the hardware features of ARMv8.1+, while the existing qemu_cortex_a53 does not. Signed-off-by: Guoqi Xie <[email protected]>
1. ARM Virtualization Host Extension (VHE) is a feature introduced in the ARMv8.1-A architecture to optimize the performance of hypervisors by allowing them to run at EL2 (the hypervisor exception level) while behaving like they are running at EL0/EL1 (user/kernel mode) in the guest. VHE simplifies the implementation of Type-2 (hosted) hypervisors by reducing context-switching overhead between the host and guest. We add it for support Zephyr to run on EL2. 2. The VPIDR_EL2 is an ARM system register at EL2 used to provide a virtualized Processor ID to guest VMs, enabling consistent and isolated CPU identification in virtualized environments. Signed-off-by: Guoqi Xie <[email protected]>
ZVM is a hypervisor based on Zephyr, we add arm64 virtulization support with Zephyr. - Added CMakeLists.txt and Kconfig changes to enable conditional inclusion of ZVM-related files when CONFIG_ZVM is enabled. - Introduced offsets.c to define offset symbols for ZVM structures, ensuring proper alignment and memory layout for ARM64 registers and contexts. Implemented ARM64 virtualization exception handling and context switching in hyp_entry.S, hyp_vector.S, and switch.c: - hyp_entry.S: save and restore vCPU register contexts and handle guest entry/exit logic. - hyp_vector.S: Defined exception vector tables for various exception types at different privilege levels, including IRQs and synchronous exceptions. - switch.c: Implemented ARM64-specific exception handlers, virtual interrupt handling, and memory access emulation for virtual devices. - Updated kernel_arch_data.h to include definitions for ZVM-specific structures like zvm_vcpu_context_t and vcpu_t. - Added zvm_offsets_short_arch.h to define macro offsets for ZVM-specific structures, facilitating memory layout calculations in assembly code. Signed-off-by: Guoqi Xie <[email protected]>
1. cpu.c: - Implemented vCPU initialization, including system register initialization, context saving/restoring, and interrupt handling for ARM64 virtualization. - Added support for vGIC (virtual GIC) and virtual timers (vtimers) for ARM64. - Included context-switching logic for guest and host system registers in hypervisor mode (EL2). - Integrated hardware feature detection to ensure virtualization support (e.g., checking for Hyp mode, VHE, and GICv3). - Supported emulation of ARM64-specific system registers (e.g., CNTV_CTL_EL0, CNTV_CVAL_EL0) and handling of virtual interrupts. 2. timer.c: - Implemented support for virtual timer initialization, including virtual timer context setup and timeout mechanisms. - Added interrupt service routines (ISRs) for virtual and physical timers to handle vCPU timer events. - Integrated logic for emulating guest timer register accesses and managing virtual interrupt propagation. - Included initialization routines for connecting vtimer and ptimer interrupts to the Zephyr IRQ subsystem. - mmu.c: Implemented stage-2 memory management for VMs, including mapping/unmapping memory, translation table allocation, and memory domain management. Signed-off-by: Guoqi Xie <[email protected]>
1. isr_wrapper.S: add gic eoi deacrive when process irq. 2. mmu.c: replace sctlr_el1 with sctlr_el2. 3. prep_c.c, smp.c: using arch_set_cpu_id_elx to set tpidr_el2. 4. reset.c: add support for hcr_el2 and enalbe VHE, support mpidr_el1, vmpidr_el2, hstr_el2, mdcr_el2, vttbr_el2. 5. intc_gicv3.c: add deactive support. 6. init.c: add vcpu_struct for thread. Signed-off-by: Guoqi Xie <[email protected]>
zvm.c: - Implemented core initialization logic for the ZVM module, including hyperviso hardware support checks, ZVM management structure setup, device initialization, and IPI handler. - Added functions for managing guest VMs, such as loading OS images and managing devices. zvm_shell.c: - Introduced shell commands for managing VMs, including creating, running, pausing, deleting, and listing VM information. - vm_manager.c: Implements VM lifecycle management, including creation (zvm_new_guest), deletion (zvm_delete_guest), and state transitions (e.g., running, paused). Adds logic for VM initialization, IRQ management, and runtime operations. - vm.c: Provides core VM handling logic, such as inter-VM communication, vCPU initialization (vm_vcpus_init), and VM state transitions. Implements functions for listing VM information and handling VCPU IRQs and context. - vm_cpu.c: Manages VCPU lifecycle, including state transitions (e.g., running, paused) and context switching. Introduces support for VCPU scheduling, SMP configurations, and CPU affinity management. - vm_mm.c: Handles VM memory management, including memory partitioning, dynamic allocation, and mapping between guest physical addresses (GPAs) and host physical addresses (HPAs). Supports memory domain creation and DTB handling for Linux VMs. Signed-off-by: Guoqi Xie <[email protected]>
This commit introduces key features for managing virtual devices and interrupts within the ZVM, enabling efficient resource allocation and IRQ management for virtual machines (VMs). - vm_device.c: Implements functions to add, remove, and manage virtual devices. Introduces memory mapping for device regions (vm_vdev_mem_add) and device-specific interrupt handling (vm_device_irq_init). Adds support for handling device emulation (handle_vm_device_emulate) and passthrough device unmapping. Also includes callback mechanisms for device IRQs and the initialization of board-specific devices. - vm_irq.c: Implements the virtual interrupt controller for managing VM interrupts. Includes functions for creating and initializing the VM interrupt control block (vm_irq_ctrlblock_create) and IRQ descriptors (vm_virq_desc_init). Adds vCPU IRQ handling logic (vcpu_wait_for_irq) and integrates IRQ initialization for devices. Provides mechanisms for mapping shared and hardware-specific IRQs across virtual and physical devices. Signed-off-by: Guoqi Xie <[email protected]>
Implements essential OS-related functions, including support for Linux and Zephyr VM. Features include OS metadata handling (get_os_info_by_type), virtual memory image loading (load_vm_image), and OS object creation (vm_os_create). These enhancements enable seamless guest OS integration and setup within the ZVM hypervisor. Signed-off-by: Guoqi Xie <[email protected]>
- vgic_common.c: Implements common VGIC functionalities, including virtual interrupt configuration, type management (virt_irq_set_type, virt_irq_get_type), priority handling (vgic_virq_set_priority), and synchronization mechanisms (virt_irq_sync_vgic, virt_irq_flush_vgic). These enhancements allow fine-grained control over virtual interrupts and efficient communication between VMs and the hypervisor. - vgic_v3.c: Adds VGICv3-specific implementations, such as initialization (vdev_gicv3_init, vgicv3_lrs_init), list register management (vgicv3_lrs_load, vgicv3_lrs_save), state save/load for interrupts, and support for SGI (Software Generated Interrupts) handling (vgicv3_raise_sgi). These features provide support for ARM GICv3-based interrupt controllers in virtualized environments. Signed-off-by: Guoqi Xie <[email protected]>
- Defined pass-through device structures in `pt_device.h`, including `pass_through_device_data` and `pass_through_device_config`, which support IRQ configuration and device-specific operations. - Implemented pass-through device initialization and IRQ handling in `pt_device_qemu_max.c`, providing a template for integrating pass-through devices with qemu_max boards. Signed-off-by: Guoqi Xie <[email protected]>
This commit introduces PSCI emulation for managing virtual CPU states and system power in ZVM. It supports PSCI 0.2, including functions like SYSTEM_OFF, SYSTEM_RESET, CPU_ON, and CPU_OFF. Signed-off-by: Guoqi Xie <[email protected]>
1. vpl011: Add a new device driver for the PL011 UART controller. 2. vserial: Add a new device driver for the virtual serial port. Signed-off-by: Guoqi Xie <[email protected]>
- thread.c: set spsr register to EL2 mode. Whitout this, thread can not switch when Zephyr run EL2. - ipi.c: add zvm ipi handler when ipi occur. - sched.c: add logic for vCPU thread switch. Only vCPU need to switch in or switch out that will call do_vcpu_swap function. Signed-off-by: Guoqi Xie <[email protected]>
@GueqiXie - please read https://docs.zephyrproject.org/latest/contribute/contributor_expectations.html A single PR with over 11 thousand lines of code is likely not going to receive a lot of attention (or sufficient attention, from another perspective). Often what I do when I have a large change, is keep the large change in draft mode. And make a separate PR for logical groups of commits. There is a natural dependency ordering, so this forces the dependency commits to be reviewed first. Once those are merged, the original draft PR can be rebased. That process can be repeated until the entire changeset has been thoroughly reviewed and merged. |
You can follow the README file to boot ZVM and different VM. Fix some Compliance Checks problems. Fix qemu_max//smp defconfig and dts. Signed-off-by: Guoqi Xie <[email protected]>
Thank you for your suggestion. I will look into whether there is a better way to structure the PR. The reason I included all the code in a single PR is that these parts are all core components for implementing virtualization in Zephyr. Without any of them, the virtualization functionality cannot be properly validated. |
You need to flip this around: virtualization might need all the commit but many of them are likely useful on their own, so you can work on submitting them separately ; for example, the introduction of qemu_max board |
This PR introduces ZVM (Zephyr-based Virtual Machine), a Type 1.5 hypervisor based on Zephyr RTOS, which allows running multiple virtual machines (VMs) on ZVM.
The sample provides support for running both Zephyr and Linux VMs on ZVM using qemu_max board. To build and run the ZVM, refer to the documentation in this PR for detailed instructions.
The project is led by Professor Guoqi Xie at Hunan University, China, with contributions from the openEuler SIG-Zephyr team and other collaborators.
For more information, refer to the project documentation and the ZVM Main Page https://gitee.com/openeuler/zvm__.